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Autonomous  Uninhabited  (or  Unmanned)  Air  (or  Aerial)  Vehicles  (UAVs)  have 
grown  in  significance  both  in  space  exploration  missions  and  in  military  applications 
such  as  surveillance  and  payload  delivery.  The  tasks  that  UAVs  are  expected  to  perform 
are  also  growing  in  complexity.  One  plausible  scenario  is  that  of  multiple  vehicles  of 
various  types,  capabilities,  and  constraints  performing  a  complicated  task  requiring 
coordinated  decision  making  and  execution. 

UAV  task  scheduling  can  be  considered  a  general  case  of  the  Capacitated  Vehicle 
Routing  Problem  with  Time  Windows  (VRPTW).  The  basic  version  of  the  vehicle 
routing  problem  is  the  Capacitated  Vehicle  Routing  Problem  (CVRP).  The  CVRP  is 
described  as  follows: 

n  targets  must  be  served  from  a  unique  depot.  Each  target  needs  a  quantity  q,  of 
ammunition  (i  -  1,  ....  n)  and  a  vehicle  of  capacity  Q  is  available  to  deliver 
ammunition.  Since  the  vehicle  capacity  is  limited,  the  vehicle  has  to  periodically 
return  to  the  depot  for  reloading.  In  the  CVRP,  it  is  not  possible  to  split  goal  delivery. 
Therefore,  a  CVRP  solution  is  a  collection  of  tours  where  each  goal  is  visited  only 
once  and  the  total  tour  demand  is  at  most  Q.  This  definition  may  be  extended  to  m 
dissimilar  vehicles  that  are  cooperating  to  serve  the  targets. 

In  general,  there  are  two  different  approaches  to  solving  this  problem,  exact  solution 
techniques  and  approximate  solution  techniques.  Various  approaches  are  considered  in 
the  literature  for  finding  exact  solutions.  These  approaches  are 

•  Dynamic  Programming 

•  Lagrange  Relaxation-Based  Methods 

•  Column  Generation 

•  Branch  and  Bound  (e.g.,  CPLEX) 

Exact  techniques  are  found  inadequate  for  real-world  problems  of  realistic  size  due  to 
computational  complexity,  and  they  often  are  complemented  with  heuristics.  Various 
approximate  solution  techniques  have  been  attempted  with  various  degrees  of  success. 
Among  these  techniques  are 

•  Directed  local  search 

•  Simulated  annealing  and  tabu  search 

•  Evolutionary  Algorithms  (e.g.  Genetic  Algorithms) 

The  problem  with  these  methods  is  that  they  require  initial  feasible  solutions 
(sophisticated  heuristics  are  often  developed  just  to  generate  an  initial  feasible  solution) 
and  have  difficulties  handling  constraints,  especially  time-windows.  These  methods 
suffer  from  slow  rates  of  convergence.  To  treat  larger  instances,  or  to  compute  solutions 
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faster,  heuristic  methods  must  be  used.  Among  the  best  heuristic  methods  are  tabu  search 
and  large  neighborhood  search.  These  approaches,  however,  generally  suffer  from  slow 
convergence. 

Another  approach  is  mathematical  programming.  In  this  approach,  the  problem  is 
transformed  into  a  Mixed  Integer  Linear  Programming  (MILP)  formulation.  The 
resulting  MILP  problem  poses  significant  challenges  to  MILP-based  solution  techniques. 
Although  much  research  has  been  devoted  to  finding  an  effective  method  for  solving 
MILP  problems,  a  general  technique  capable  of  handling  online  task  scheduling  has  been 
elusive  due  to  the  potential  for  exponential  explosion  in  large-scale  problems. 

Mixed  Integer  Nonlinear  Programming  (MINLP)  formulations  cover  a  broad  class  of 
problems,  including  MILP,  and,  in  general,  are  more  difficult  to  solve  than  MILP 
formulations.  A  wide  variety  of  optimization  problems  arising  in  engineering 
applications  can  be  formulated  as  MINLP  problems.  Applications  of  MINLP  range  from 
control  of  hybrid  systems  [1]  to  task  scheduling  [2]  and  other  closely  related 
combinatorial  optimization  problems,  such  as  the  traveling  salesman  problem  (TSP)  and 
path  planning  [3][4],  All  these  problems,  however,  pose  significant  challenges  to 
MINLP-based  solution  techniques.  Although  much  research  has  been  devoted  to  finding 
an  effective  method  for  solving  MINLP  problems,  a  general  technique  capable  of 
handling  real-time  task  scheduling  as  well  as  assignment  and  control  problems  has  been 
elusive.  This  is  due  again  to  the  potential  for  combinatorial  explosion  in  large-scale 
problems  [5][6]. 

The  deficiencies  in  applying  any  stand-alone  optimization  technique  have  led 
researchers  to  attempt  to  combine  efficient  components  of  the  different  methods  to  build 
hybrid  optimization  solutions.  Integration  of  different  features  of  multiple  conventional 
optimization  and  heuristic  techniques  is  an  area  that  has  sparked  considerable  interest  in 
the  optimization  community.  The  main  motivation  behind  these  endeavors  is  the  fact 
that,  with  the  increase  in  the  dimension  of  the  problem,  the  conventional  methods  quickly 
become  computationally  intractable.  The  standard  techniques  for  solving  MILP 
problems,  e.g.,  branch  and  bound,  experience  an  exponential  rise  in  convergence  time 
when  the  number  of  variables  is  increased  [6],  This  rise  in  complexity  with  the  increase 
in  dimension  is  evident  even  for  a  moderate  increase  in  problem  size. 

Grossman  and  Jain  [7]  have  presented  a  hybrid  MILP  and  constraint  programming 
(CP)  techniques  to  solve  problems  that  are  intractable  if  solved  using  either  of  the  two 
techniques  alone.  The  class  of  problems  they  tested  was  formulated  in  terms  of  a  hybrid 
M1LP/CP  model  that  involves  some  of  the  MILP  constraints,  a  reduced  set  of  CP 
constraints  and  an  equivalence  relation  between  the  MILP  and  CP  variables.  Fletcher  [8] 
has  presented  another  approach  by  outer  approximation.  The  outer  approximation  scheme 
solves  MINLP  problems  by  a  finite  sequence  of  Nonlinear  Programming  (NLP)  sub¬ 
problems  and  MINLP  master  problems.  Other  interesting  attempts  at  hybrid  solutions 
combine  some  conventional  methods  with  heuristic  or  metahueristic  approaches  in  the 
hope  of  having  their  strengths  complement  one  another.  Salomon  [9]  has  combined 
gradient  search  methods  with  evolutionary  techniques  to  achieve  faster  convergence  on 
continuous-parameter  optimization  problems. 


2 


In  our  work,  we  present  a  new  hybrid  method  for  solving  MINLPs  for  a  class  of 
combinatorial  optimization  problems  formulated  as  follows: 


min^  3(x) 

subject  to  :  B  <  Cx  <  B 
Where  x  - 1 


fx  ' 


\xdJ 


zW'-k  xde{-\,\)k 
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In  the  following  sections,  the  Hybrid  Projected  Gradient-Evolutionary  Search  (HPGES) 
algorithm  that  was  developed  to  solve  this  problem  is  described  in  detail. 

1  Hybrid  Projected  Gradient-Evolutionary  Search  Algorithm 

The  HPGES  algorithm  uses  a  specially  designed  evolutionary-based  global  search 
strategy  to  efficiently  create  candidate  solutions  in  the  solution  space.  A  local  projection- 
based  gradient  search  algorithm  is  then  used  to  improve  the  candidate  solutions  at  each 
generation  and  to  construct  new  (potentially  improved)  candidate  solutions  for  the  next 
generation  of  the  evolutionary  search.  The  search  terminates  when  a  certain  convergence 
criterion  is  met.  The  details  of  different  components  of  the  hybrid  algorithm  are  described 
in  detail  in  the  following  sections. 

1.1  Discrete  Variables  Relaxation 

The  generalized  MINLP  can  be  stated  as: 

min,  3(x) 

subject  to :  B<Cx<  B 


where  x  is  the  decision  vector  with  xc  and  xd  indicating  continuous  and  discrete  decision 
variables,  respectively.  B  and  B  are,  respectively,  the  lower  and  upper  bounds  for  the 
constraint  set  Cx  where  C  is  a  matrix  of  appropriate  dimensions.  We  assume  that  xeQ, 
where  £1  is  a  bounded  space. 

A  class  of  algorithms  for  solving  Eq.  (1)  is  based  on  penalty  function  methods. 
Among  the  earliest  such  methods  is  the  one  proposed  in  [10],  where  the  authors  observed 
that  a  binary  variable  xd  j  could  be  replaced  by  a  continuous  unbounded  variable  if  an 

appropriate  constraint  is  added.  Even  though  the  reformulation  in  [10]  appears  attractive, 
it  suffers  from  a  number  of  serious  drawbacks.  Introducing  a  nonlinear  equality 
constraint  transforms  a  convex  MINLP  problem  into  a  non-convex  NLP  problem,  which 
in  general  is  not  easier  to  solve.  Another  approach  aimed  at  relaxing  the  MINLP  problem 
into  a  NLP  problem  is  suggested  in  [1],  where  use  of  a  penalty  function  and  a  constraint 
was  advocated: 
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subject  to :  B<Cx<  B 
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In  Eq.  (3),  x  is  the  same  as  in  Eq.  (2),  where  xd  =  [xd  ,,..., xd  m]  is  the  relaxed 
binary  decision  variable,  where  the  xd  j ,  /  <7  <7n,  are  expected  to  converge  to  0  or  1 


(the  positive  penalty  term 


/=! 


xdxd-'YJxd  i  will  have  the  minimum  value  of  zero  when 


xd j ,  ]  <i  <m,  are  either  0  or  I)  and  non-negative  cr,  and  a2  are  appropriately  selected 
at  each  iteration.  The  algorithm  repeatedly  solves  the  optimization  problem  in  Eq.  (3) 
with  an  increasing  penalty  factor  a,  and  tighter  tolerance  level  a,  until  the  difference 
between  two  successive  iterations  becomes  sufficiently  small  and  the  elements  of  the 
solution  vector  approach  0  or  1.  If  the  algorithm  converges  to  a  solution  whose 
components  are  not  0  or  1,  then  the  procedure  is  restarted  from  an  alternative  initial  point. 
Other  forms  of  penalty  function  are  also  suggested  in  the  literature. 


Relaxing  binary  variables  through  a  combination  of  augmented  penalty  functions  and 
additional  constraints  has  been  found  to  suffer  from  two  major  setbacks.  First,  in  general, 
extraneous  local  optima  will  be  introduced  to  the  optimization  surface  within  the  feasible 
region,  and  hence  the  true  optimal  solution  could  be  compromised.  Second,  the  solution 
for  the  relaxed  binary  variable  will  converge  to  0  or  1  only  when  the  penalty  factor,  cr,  in 
Eq.  (.3)  for  instance,  approaches  infinity.  For  the  quadratic  cost  function  in  Eq.  (3), 
however,  the  use  of  cx(xrf— 0.5)  (*rf-0.5)  (where  0.5  is  the  vector  all  of  whose 
elements  are  0.5)  as  the  augmented  penalty  function,  and  the  addition  of  0<xrf,<l, 

1  <i  <m,  as  the  additional  constraint,  is  found  to  be  adequate  even  with  moderate  values 
of  cr .  This  is  the  relaxation  strategy  adopted  in  the  solution  to  the  generalized  MINEP 
with  the  graph  partitioning  application  that  is  studied  in  this  work,  transforming  a  M1NLP 
problem  into  a  constrained  NLP  problem  to  be  solved  with  the  HPGES  algorithm. 


1.2  Projected  Variable  Metric  Method 

Unconstrained  NLP  problems  have  been  studied  extensively  in  the  past,  and  a  host  of 
well-known  techniques,  including  conjugate  direction  methods,  restricted  step  methods, 
Newton-like  methods,  and  secant  methods,  is  reported  in  the  literature  (see  [8],  [11]  and 
[12]  for  instance).  The  convergence  rate  of  these  methods  is  super-linear,  and  global 
convergence  is  guaranteed  for  convex  and  smooth  functions.  The  solution  to  constrained 
NLP  problems,  however,  becomes  prohibitively  expensive  for  large-scale  problems.  This 
work  advocates  a  projection  technique  that  has  demonstrated  favorable  numerical 
properties  in  the  solution  to  the  generalized  MINLPs.  [13], 
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1.3  Hybrid  Evolutionary-Gradient  Algorithm 

In  this  section,  a  hybrid  evolutionary-gradient  based  method  for  MINLP  problems 
with  inequality  constraints  is  introduced.  In  this  approach,  the  evolutionary  algorithm  is 
used  as  an  engine  for  global  search,  while  the  projected  variable  metric  method  is  used 
for  efficient  and  swift  local  search. 

A  local  search  algorithm  starts  with  an  initial  solution  and  seeks  better  solutions  in  the 
existing  candidate  solution's  neighborhood.  The  quality  of  the  solutions  obtained  by  a 
search  method  is  directly  influenced  by  the  definition  of  the  neighborhood  space  in  the 
search  process.  Efficient  generation  of  superior  neighborhoods  is  important  in  order  to 
have  an  effective  search.  Being  trapped  in  local  optima  is  the  other  factor  affecting  the 
quality  of  solutions  generated  using  local  search  methods.  Combinations  of  local  search 
techniques  with  global  search  strategies  such  as  genetic  algorithms  were  devised  to 
overcome  such  problems.  These  search  strategies  are  classified  as  metahueristics  since 
they  act  as  guidance  strategies  for  their  respective  local  search  procedures. 

Genetic  algorithms  (GAs)  are  a  class  of  guided  search  heuristics  for  the  optimization 
process  based  on  the  gradual  evolution  of  genes  in  a  population  of  solutions  represented 
as  chromosomes.  Candidate  solutions,  or  chromosomes,  are  usually  fixed-length  integer-, 
real-  or  binary-valued  strings.  A  selection  mechanism  chooses  parent  chromosomes  to  go 
through  recombination  and  mutation  procedures  possibly  to  produce  better  candidate 
solutions.  Improved  solutions  may  replace  some  unfit  members  of  the  old  population. 

The  steps  involved  in  combining  the  two  aforementioned  methods  to  form  the  Hybrid 
Projected  Gradient-Evolutionary  Search  (HPGES)  algorithm  are  as  follows  (see  Figure 
1): 

1.  Create  a  random  initial  population.  Use  a  Linear-programming  solver  (e.g., 
CPLEX)  to  ensure  that  at  least  some  of  the  individuals  in  the  initial  population  are 
feasible.  (This  would  not  be  the  case  for  a  totally  random  initialization.) 

2.  Create  new  individuals  by  using  the  local  improvement  scheme  (gradient 
projection)  on  the  population  of  the  candidate  solutions.  Infeasible  individuals  are 
also  made  feasible  through  the  projection  process. 

3.  Select  individuals  from  the  pool  of  improved  feasible  candidate  solutions  for 
crossover. 

4.  Apply  convex  crossover,  create  new  candidate  solutions,  and  update  the 
corresponding  search  direction  matrix  for  each  newly  created  individual. 

5.  Select  the  individuals  in  the  population  pool  for  the  next  iteration. 

6.  Stop  if  convergence  criterion  is  met.  Otherwise,  go  to  step  2. 
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Figure  1:  Hybrid  Projected  Gradient- Evolutionary  Search  Algorithms  for  MINLPs 

To  create  the  initial  population  with  random  feasible  solutions,  we  use  the  LP  solver 
of  CPLEXR  and  solve  the  following  LP  problem  repeatedly  with  a  random  vector  L, : 

min^  L,x 

subject  to  :  B<Cx<  B 

The  main  idea  in  the  improvement  stage  (see  Figure  1)  is  to  use  the  local  search 
algorithm  in  order  to  swiftly  improve  the  candidate  solutions  in  the  population  pool.  The 
improved  feasible  solutions  are  then  presented  to  a  selection  and  diversification  operator 
(in  this  case,  the  convex  crossover  of  the  selected  individuals  in  the  population)  in  order 
to  explore  the  solution  space.  A  variety  of  GA  operators  can  be  used  at  each  stage  (see 
[14]  for  details). 

1.4  Numerical  Implementation  Results 

Although  our  algorithm  provides  a  general  framework  for  a  wide  range  of  problems 
formulated  as  MINLPs,  the  initial  tests  were  done  on  sample  graph  portioning  problems, 
and  the  results  were  compared  to  best  reported  results  from  the  literature.  Many  real- 
world  problems  of  interest  can  be  formulated  as  graph  portioning  problems,  which  in  turn 
can  be  cast  into  a  formulation  suitable  for  the  proposed  HPGES  algorithm.  The  results 
discussed  in  the  following  section  outline  the  utility  of  the  algorithm.  These  results  can 
probably  be  improved  for  particular  applications  by  incorporating  heuristic  schemes 
suitable  for  each  problem. 
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1.5  The  Graph  Partitioning  Problem 

The  proposed  algorithm  for  MINLP  is  tested  on  the  multiset  min-cut  graph 
partitioning  problem.  In  this  problem,  the  vertices  of  a  graph  are  partitioned  into  sets  of 
given  sizes  while  minimizing  the  weighted  sum  of  the  cut  edges,  that  is,  the  edges 
connecting  vertices  in  different  sets.  Let  A  be  an  n-by-n  weight  matrix  associated  with  a 
directed  graph  with  vertex  set  V  =  {1,  2, ...  n}.  Multiset  min-cut  graph  partitioning  can  be 
represented  as  follows: 

Given  mt ,  l<i<k ,  partition  V  into  k  disjoint  subsets  so  that 

V  —  V \  Lt  V2  V3  ...U  Vk  5  |F,  |  =  mi  y  /  </  <k,  in  a  way  that  the  sum  of  the 
weights  of  the  cut  edges  is  minimum.  For  implementing  the  algorithm,  we  used 
random  graphs  with  n  vertices,  where  an  edge  with  a  random  weight  between  any 
two  vertices  is  created  with  probability  p  =  \/Jn.  In  partitioning,  we  assumed  that 

all  1  <  i<k,  are  equal.  We  also  tested  the  algorithms  on  graphs  from  standard 
test  datasets.  The  simulation  software  has  been  written  in  MATLAB,  and  the 
coded  algorithm  has  been  tested  on  a  Pentium  4,  2.2  GHz  CPU  with  1  GB  RAM. 

(For  results  refer  to  Table  I .) 

2  Simulation  Results  and  Discussion 

The  simulation  results  for  randomly  generated  and  standard  benchmark  problem 
instances  are  presented  in  Table  1 .  In  all  the  instances,  the  HPGES  algorithm  was  able  to 
converge  to  optimal  or  near  optimal  solutions  within  the  iteration  count  specified.  The 
time  taken  in  convergence  for  the  main  hybrid  routine  is  also  tabulated  in  Table  I.  The 
results  indicate  that  the  growth  in  CPU  time  is  not  exponential  in  the  growth  in  problem 
size.  The  results  are  also  encouraging  in  the  sense  that  the  randomly  generated  graphs  are 
generally  ill  conditioned  and  do  not  possess  a  structure  particularly  amenable  to 
numerical  implementation.  Some  heuristic-based  methods  for  graph  partitioning  could 
also  be  included  to  improve  the  results  and  reduce  CPU  time.  However,  the  objective  of 
these  examples  is  to  evaluate  the  utility  of  the  proposed  algorithm. 


Problem  size 

Problem  type 

Maximum  Iterations 

Number  of  cuts 

Elapsed  Time  (sec) 

20 

Random 

20 

100 

17.468 

40 

Random 

20 

400 

43.672 

60 

Random 

20 

900 

90.077 

80 

Random 

20 

1600 

166.888 

136 

Benchmark 

20 

11 

284.965 

548 

Benchmark 

20 

47 

546.236 

Table  1:  Simulation  Results  with  Running  Times  for  the  Graph  Partitioning  Problem 
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3  Conclusion  and  Future  Work 

To  solve  the  online  scheduling  problem  of  cooperating  UAVs,  we  developed  a  new 
Hybrid  Evolutionary  Gradient  Projection  algorithm  for  mixed  integer  nonlinear 
optimization  problems  with  linear  inequality  constraints.  The  core  engine  of  the 
algorithm  is  a  novel  local  improvement  scheme  based  on  projected-gradient  search.  The 
evolutionary  mechanism,  on  the  other  hand,  guides  the  solution  search  out  of  local 
optima,  towards  regions  closer  to  global  optima.  The  projection  algorithm  enables 
simultaneous  addition  or  removal  of  multiple  constraints  to  or  from  the  active  constraint 
set,  increasing  the  speed  of  convergence.  The  penalty  function  used  to  relax  the  integer 
variables  is  shown  not  to  introduce  any  additional  local  optima  inside  the  feasible  region, 
hence  avoiding  one  of  the  main  deficiencies  of  the  similar  relaxation  attempts  in  the  past. 
The  modified  cost  function  and  the  appropriately  constructed  projection  mechanism 
guarantee  a  feasible  solution.  The  evolutionary  component  of  the  hybrid  algorithm 
maintains  diversity  of  the  search  regions,  hence  improving  the  possibility  of  finding  a 
solution  at  or  near  a  global  optimum  in  a  reasonable  time.  The  gradient  search 
component,  on  the  other  hand,  significantly  speeds  up  the  convergence  of  the  hybrid 
algorithm,  compensating  for  the  slow  convergence  of  the  evolutionary  search.  The 
hybrid  algorithm  also  requires  a  smaller  population  as  compared  to  conventional,  pure 
evolutionary  algorithms.  Efforts  to  compare  thoroughly  the  numerical  properties  of  this 
new  algorithm  with  those  of  other  available  M1NLP  solvers  are  currently  underway. 
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